Improving Compiler and Run-Time Support for Irregular Reductions

Compilers for distributed-memory multiprocessors parallelize irregular reductions either by generating calls to sophisticated run-time systems or relying on the sharedmemory interface supported by software DSMs. Run-time systems gather/scatter nonlocal results (e.g., CHAOS, PILAR) while software DSMs apply local reductions to replicated buffers (e.g., CVM, TreadMarks). We introduce LOCALWRITE, a new technique for parallelizing irregular reductions based on the owner-computes rule. It eliminates the need for buffers or synchronized writes, but may replicate computation. We investigate the impact of connectivity (node/edge ratio), locality (accesses to local data) and adaptivity (edge modifications) on their relative performance. LOCALWRITE improves performance by 50–150% compared to using replicated buffers. Gather/scatter using CHAOS generally provides the best performance, but LOCALWRITE can outperform CHAOS for applications with low locality or high adaptivity. We also discover the flushupdate coherence protocol can improve performance by 15– 25% for software DSMs over an invalidate protocol.